Computer network data distribution and selective retrieval system

ABSTRACT

A method of accessing a distributed virtual world such that the only data requested of the network data sources and transmitted to the requesting computer is the data that codes for objects that the viewer can perceive, and to prioritize the data to be sent in order of its perceivability so that only relevant data is sent to the user.

FIELD OF THE INVENTION

This invention relates generally to a computer network data distributionand retrieval system capable of enabling the creation and presentationof a large, variably detailed, three-dimensional virtual reality worldwith data stored on distributed network sources and obtained by computernetwork transmission. A virtual world created using the datadistribution and retrieval system of the present invention may beaccessed by one or more users concurrently. The invention furtherrelates to a method of user access to this virtual world, and a methodof user interaction with objects and with other users in the world.

BACKGROUND OF THE INVENTION

The creation of virtual reality (VR) worlds using data stored on sourcesdistributed across a computer network, such that individual computerusers may “enter” the VR world by accessing the computer network, hasbeen a longtime goal of computer scientists. Of particular interest areVR worlds that may reside on wide area computer networks such as theInternet.

Although hypothetical VR worlds that users may enter and interactwith—and which are indistinguishable from actual reality—have been shownin television shows such as “Star Trek: The Next Generation” and “MadAbout You,” these representations, like most popular representations ofVR technology, merely use videotape special effects to create anillusion of a virtual world. In contrast, a true virtual world is one inwhich every object, including a user's image, is digitally coded andrendered.

A more authentic virtual world was used as a critical dramatic device in“Disclosure,” the recent movie based on the Michael Crichton novel ofthe same name. In that world, a user could retrieve data from electronicfiles by entering a digitally rendered filing room (representing a datastorage device), opening a rendered filing cabinet (directory), andremoving rendered sheets of paper (documents) stored in rendered manilafolders (folders). Further, the user saw a digital image of himself andother users in the world. Like most of Crichton's work, thisrepresentation of VR technology was just barely fictional.

However, to create its great level of user immersion in the VR world,the fictional VR system in “Disclosure” required that the user don aspecial full-body suit equipped with goggles and motion/positionsensors. Further, the user could only access the world via a specialgimbaled platform to which the suit was attached. Indeed, the use ofgoggles, gloves, and other apparatus is considered to be a necessarycondition of virtual reality. It is understood for the purposes of thepresent invention that the discussion pertains to two-dimensionalrepresentations of a three-dimensional environment. Thus, the presentinvention relates to a virtual world that may be accessed solely via aflat display screen, although the use of other interface devices such asgoggles, gloves, or VR suites is not precluded.

While recent advances in computer hardware and telecommunicationstechnology have enabled the creation of rudimentary virtual worlds thatmay be accessed via a computer network, there remain several significantobstacles to creating a world in which motion is experienced inreal-time and in which users can interact with inanimate objects in theworld and with other users. The most significant problem with creating alarge virtual reality world is that a world sufficiently large anddetailed to be interesting contains more data than can be transmitted tothe user's computer in a reasonable amount of time. In addition, evenwith the tremendous advances in microprocessor speed, network speed, anddata storage capability of personal computers, the amount of data anddata processing required to create an interesting virtual worldoverwhelms currently available systems.

Previous attempts to create a large virtual reality world on a computernetwork, specifically the Internet, have been unable to clear thistechnological hurdle and accommodate the extensive data that current VRtechnology requires. Two current approaches to the problem that fail indifferent ways are current VRML implementation and Alpha World.

Currently, the most popular method of sending three-dimensional scenesacross the Internet is using the Virtual Reality Modeling Language(VRML). VRML's operating principle is that every object located in aparticular scene is described fully by data contained in an ASCII textfile. When a user wants to enter this scene, under currentimplementation the entire file is transferred en masse over the networkto the user's computer at one time. An application program on the user'scomputer can then access and manipulate the text contained in this fileto render the three-dimensional scene. Once the scene has been rendered,the user can move around in this world and interact with objectscontained within it.

There are several problems with the current VRML implementations. Thegreatest flaw is that data is not efficiently transferred over thenetwork. Also, the data storage and processing limitations of currentlyavailable personal computers limits the complexity of the scene.Finally, current VRML implementations require the user's computer to doa lot of unnecessary work.

These current VRML implementations are inefficient in its use theirnetwork resources because they do not differentiate between relevant andirrelevant information. Each VRML file contains all of the datanecessary to render completely every aspect of a particular scene.Although some of the objects coded by the data will be seen by the userat any one time, many more of the coded objects either are too far awayfrom the user to be visible, are outside the user's cone of vision andtherefore not visible, or are hidden from the user's view behind otherobjects. Thus, a lot of valuable time and resources are spent ontransferring to the user's computer and subsequent processing of datathat is unused and unnecessary at any given time.

Also, the large data files needed to represent all objects in an entirescene take a long time to transfer over the network. The long transfertime causes the user to experience long data transfer delays whenentering each new scene. Further, once the data files transferred, theyare unwieldy for even the fastest computers to process. Thus, there areoften long processing delays when the user moves about the scene. Theseseemingly interminable delays significantly detract from the virtualreality experience.

Another problem with the current VRML implementations is that networkdelays grow rapidly as the complexity of a scene is increased. If ascene has twice as much detail (e.g., twice as many objects), it willtake twice as long to transfer across the network the data needed tocreate the scene. The scene designer faces an intractable problem whendetermining how much detail to put into a given scene. If too littledetail is included, users who have fast network connections or who arewilling to tolerate delays will not get as much graphic data as theywould like. The virtual world is not as interesting to them. But, asmore detail is included to make the scene interesting to this firstgroup of users, other users with slower network connections or withcomputers that cannot render highly detailed worlds will be forced towait a long time to receive information that their computers cannotdisplay at an acceptable speed. Thus, regardless of how much detail isincluded in the scene, someone is bound not to have an enjoyable VRexperience.

Also, these current VRML implementations place more demands on theuser's computer than is necessary. First, the user's computer must storeall of the data contained in each VRML file either in active memory(RAM) or cache it to disk, both of which slows the processing. Evenworse, before rendering the scene, the computer must prioritize the datato determine what parts of the scene should be displayed, discarding theirrelevant data. This can be a time consuming process. Very few computerusers today have enough RAM to easily store and manipulate all of thedata needed to create a reasonably complex three-dimensional scene.Further adding to the problem, the computer cannot begin to do anyprocessing until after the entire VRML file has been received. It canneither display a good approximation of the scene or begin to processthe entire scene until after the file transfer is completed.

In conclusion, the current VRML implementations not the best way toprovide three-dimensional worlds over a computer network because it usesexcessive network bandwidth, it wastes computer resources, and it limitsthe complexity of the scenes that can be represented.

Alpha World is another attempt at creating a three-dimensional world onthe Internet. Unlike the current VRML implementations, Alpha World doesnot send all at once all of the data needed to create the entire VRworld. However, Alpha World has other deficiencies that detract from aninteresting VR experience.

Alpha World's approach is to divide the world into a grid of squareregions. Only a few regions to each side of a viewer are retrieved fromthe network and displayed at any one time. This method allows the worldto be arbitrarily large. However, there are many problems with AlphaWorld's method; most of which relate to how it uses the network. Thespecific problems can be divided into two categories, Data Retrieval andData Processing.

The first problem with Alpha World's use of the network is in the actualdata it requests. Specifically, Alpha World will request from thenetwork objects that cannot be seen. Also, Alpha World fails to requestobjects that can be seen. For example, when a person walks down a streetin Alpha World, the road appears to come to an end in front of him. Hecan not see into the distance. However, the program has spent valuabletime requesting objects located inside a house that is to the left ofthe person and that cannot be seen.

The problem is most apparent when the user elevates above the world andlooks down, taking a bird's eye view. All the user sees is a small patchof streets and buildings directly below. There are no objects aroundthis patch or on the horizon. However, the objects in this patch areunnecessarily detailed. The program spends time getting detail for smallobjects, while large objects that should be seen are not sent over thenetwork. Alpha World could solve this problem by showing the objectscontained in more squares of the grid. However, in order to maintain thesame amount of data to be sent over the network, each square would haveto have fewer objects. The result is that although Alpha World can bearbitrarily large, it cannot also be highly detailed.

This inefficient use of the network also leads to a problem in userinteraction with the virtual world. When a new grid square gets closeenough to the user, Alpha World requests from the network all of thedata for all of the objects residing in that square. When these objectsare received, they are displayed all at once. This effect leads tochoppy movement of objects on the display screen, which detracts fromthe VR experience and makes it difficult for the user to interact withthe objects in the world. Also, the objects are cached locally all atonce when they are received. This procedure takes time and causes theentire program to pause for a couple of seconds. During this pause it isimpossible for the user to interact with other people or objects in thevirtual world.

Another problem with Alpha World is the time spent processing data forall of the hidden and irrelevant objects that it receives from thenetwork. This inefficient use of processing power forces Alpha World tomake sacrifices in the complexity of the world.

One such sacrifice is height. Although Alpha World looks like athree-dimensional world, it is actually flat. Objects cannot be stackedon top of each other. The term used to describe such a world is “two anda half dimensional.” Alpha World is not able to process enoughinformation for a fully three-dimensional world because it spends toomuch time processing hidden objects. Also, Alpha World does not alwaysdisplay the world correctly. Sometimes objects in the distance aredisplayed in front of objects that are actually in the foreground.

This effect occurs because Alpha World does not z-buffer. Nor couldAlpha World do this, as z-buffering would consume too much processingpower since the data that codes for hidden objects would still need tobe processed.

Another problem caused by Alpha World's inefficient processing is thatall objects must be simple shapes, since complex objects with many faceswould require too much processing power to render. Thus, most of theobjects in Alpha World are simple bitmaps which look the same from allangles. In addition, every object must fit within the square blocks. Inorder to build a larger object, these blocks must be pieced together.This limitation makes it difficult to create objects oriented at anangle. The result is that a town in Alpha World will look like it islaid out on a grid.

Other approaches to storing and retrieving information in a distributednetwork also have technological drawbacks that create user problems. Forexample, a major problem with the World Wide Web is that its serversoften become overloaded. This overloading slows down the interactionwith every user using that server. Physically, there is no need for thisproblem to occur. For every server that is overloaded, there are manyother servers that are doing nothing, or very little. However, in thepast, there has been no mechanism for distributing the load from theoverburdened servers to these other unused or underused computers. Whatis needed is a method of propagating objects around the network in ordersolve the problem of server overload. In such a distributed,decentralized system, there would be no servers or clients. Eachcomputer in the system would run a program that acts as a node.

In order to create a large virtual reality world for use on such acomputer network, a new method is needed to transfer and process theenormous amount of data necessary to create a world that is both usefuland interesting to the user. Because an interesting scene contains toomuch data to transmit all at once, the virtual world should be arrangedsuch that detail for larger objects is represented by smaller objects.The goal should be to send the detail for only those objects that aremost relevant to the user at each point in time. A good way to measurethe relevance of an object is to measure its perceivability to the user.

Accordingly, it is an object of this invention to provide a computernetwork data distribution and retrieval system wherein each computer inthe system acts as a node, thereby enabling perceivable objects to bepropagated across the network and eliminating the incidence of serveroverload.

It is another object of the present invention to provide for creation ona computer network of a large and detailed virtual reality world that isboth useful and interesting to the user.

It is a further object of the present invention to create a method ofaccessing a distributed virtual world such that the only data requestedof the network data sources and transmitted to the requesting computeris the data that codes for objects that the viewer can perceive, and toprioritize the data to be sent in order of how well it can be perceivedso that only relevant data is sent to the user.

It is another object of this invention to provide a computer networkdata distribution and retrieval system wherein detail for larger objectsis represented by smaller objects in order to facilitate sending acrossthe network only the objects that the viewer can perceive.

SUMMARY OF THE INVENTION

The objects of the present invention are met by a computer network datadistribution and retrieval system comprising computers that function asdata storage repositories and network nodes, and that also enable useraccess to data distributed across the network. Although the embodimentdiscussed is the use of the computer network data distribution andretrieval system of the presently claimed invention to create athree-dimensional virtual world, the present invention relates to alltypes of data that can be arranged in levels of increasing detail andfor which a perceivability index can be measured.

There are no dedicated servers or clients in the computer network datadistribution and retrieval system of the presently claimed invention.Each computer in the system runs an application program that enablesthat computer to act as a network node. When the application programneeds an object that is not stored on the local computer, it sends arequest to the remote computer that owns the object. If that remoteowner computer is not already responding to requests from too many otherrequesting computers, the owner computer transmits the object to therequesting computer across the network and adds the requesting computerto the owner computer's list of requesting computers to which it isresponding.

However, if the computer that owns the object is already responding toits maximum number of requesting computers, the owner computer removesfrom its queue one of the prior requesting computers to which it hadbeen responding. The owner computer also sends the removed requestingcomputer a list of all the other requesting computers to which the ownercomputer has been responding. The removed requesting computer can thensend its data requests for more detail about the object to the computersin that list. This process continues until the computer that can respondto requests the fastest is found. Thus, perceivable objects willpropagate around the network and every requesting computer will attemptto retrieve objects from nearby, fast computers.

Other objects, features and advantages will be apparent from thefollowing detailed description of preferred embodiments thereof taken inconjunction with the accompanying drawings in which:

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a schematic view of how the data storage and retrievalsystem of the present invention determines whether a perceivable objectis to be displayed to the user.

FIG. 2 shows a pictorial representation of variations in the level ofdetail related to an object that is presented to the user depending uponthe area of the user's visual field that the object occupies.

FIG. 3 shows a schematic representation of perceivable objectpropagation on a three-node computer network.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

In order to understand the methods used by the presently claimedcomputer network data distribution and retrieval system to achieve thetechnical objectives of the invention, it is necessary to describe howthe system creates perceivable objects and propagates them across thenetwork.

Perceivable Objects

In the computer network data distribution and retrieval system of thepresent invention, every piece of data that the user's computer networkaccess application program can present to the user is in the form of aperceivable object. These perceivable objects are identified by a URI(Universal Resource Identifier) that includes information about theremote computer that owns the object, as well as the object's locationon that remote computer.

In addition, the user's computer retains local copies of all perceivableobjects that it has requested. These local copies need not contain allof the information relevant to a given perceivable object. Instead, thelocal copies contain only the information that is necessary to performthe processing tasks currently needed by the user's computer. If a localperceivable object copy does not have certain necessary information, theuser's computer requests the information over the network and obtainsthe needed information from another copy of the perceivable object onanother network computer.

Messages

All network communication is achieved by transmission of messages amongthe networked computers. Each perceivable object is associated with onemessage for each type of interaction that may be performed on theperceivable object. Perceivable objects use messages to accomplish bothsystem functions and user interaction with the object. Each type ofmessage may contain various values that more clearly specify themessage's purpose. The use of messages for all network communicationsimplifies the design and increases the extendibility of the applicationprogram.

Network Efficiency

There are two key optimizations that enable the application program toeffectively present to the user a large and detailed interactive worldwith data distributed across a computer network. The first optimizationis the selective retrieval of information about the world. The secondoptimization is the propagation across the network of heavily accesseddata from high traffic nodes to lower traffic nodes.

Selective Data Retrieval

The perceivable objects in the world that the application programpresents to the user are arranged in a hierarchical structure that makesit possible at any given point to send across the network only thoseportions of the world that the user can perceive. Since the number ofperceivable objects that a user can perceive at any one time does notincrease with the size or detail of the world, this approach enables thecreation and use of arbitrarily large and detailed worlds.

The process begins when a single perceivable object is created. Therepresentation of this and all other perceivable objects contains asimple approximation of the structure of the perceivable object and amanifest flag indicating whether the perceivable object should bepresented to the user. This flag is initially set, indicating that theperceivable object will be presented to the user. It also contains alist of the network locations of smaller perceivable objects thattogether combine to represent the main perceivable object in higherdetail.

When it is time to present the initial perceivable object to the user,the perceivable object is processed as follows. First, itsperceivability from the user's viewpoint is calculated and is comparedto the detail threshold value. Both perceivability and the detailthreshold depend on the application program and are described below. Ifthis perceivability is less than the detail threshold value, then theperceivable object's manifest flag is set, the perceivable objects thatmake up its details are deleted, and any pending network requests by theperceivable object are canceled. If the perceivable object'sperceivability is above the detail threshold, the manifest flag istested. If this flag is set, the perceivable object is presented to theuser, and a request for its sub-objects is put into the network prioritylist. Otherwise, each of the smaller perceivable objects that make upthe current perceivable object is processed in the same way as the mainperceivable object. The flowchart in FIG. 1 shows this process.

The perceivable objects that the application program presents to theuser are linked together in a tree structure. Each representation of aperceivable object in the virtual world contains two important pieces ofinformation. The first piece of information is a simple representationof the physical structure of the perceivable object (e.g., in the caseof a geometrical object, whether it is generally a circle, square,rectangle, triangle, etc.). The second piece of information is a list ofpointers to other, smaller perceivable objects whose union comprises amore detailed representation of the structure of the main perceivableobject. These smaller perceivable objects can have further sub-objectsof their own. This hierarchical structure not only makes it possible tosend across the network perceivable object representations havingdifferent levels of detail, but also facilitates extremely efficientrendering of the world to a graphic or other display capable ofpresenting the object.

Because the world that the application program presents is so vast, theprogram must carefully choose which data it will request from thenetwork. As shown in FIG. 1, when the application program presents tothe user a VR world from the viewpoint of the user, it calculates foreach perceivable object the perceivability of that object to the user.The application program compares the calculated perceivability valuewith a certain threshold value. If an object's perceivability value isgreater than the threshold value, the application program will theninstruct the object to add the sub-objects for that object to thenetwork request queue so that detail data for those sub-objects can bepresented to the user. Once in the request queue, the sub-objects canoptionally send messages requesting more detail about themselves fromanother computer that has that detail. If an object's perceivabilityvalue is less than this threshold value, the application program willinstruct the object to delete any sub-objects for that object and removethem from the network request queue. This process is performed each timethe virtual world is presented to the user.

Referring again to the flowchart of FIG. 1, it may be understood thatthe following code operations will enable the claimed invention toaccomplish the desired objectives:

DisplayObject (Object) Visibility ← GetVisibility (Objeet) IfVisibility > Threshold If DisplayFlag PutObjectInNetworkPriorityList(Object) RenderObject (Object) Else For Each Subject (Object)DisplayObject (SubObject) Else For Each SubObject (Object) DeleteSubObject RemoveObjectFromNetworkPriorityList (Object) DisplayFlag ←True RenderObject (Object)

The network request queue prioritizes messages received from allperceivable objects. Often, there is more than one perceivable objectthat needs to use the network to inquire about its physical structure.In this case, the request generated by the perceivable object with thegreatest perceivability value is sent across the network first. Thus,the data that is most perceivable to any user is retrieved first, sothat the user's view of the world is as detailed as is possible. Whenresponses to these requests are received, new perceivable objects arecreated to represent the increased detail. If the perceivability of anyof these objects is above the threshold value, the application programwill then request more detail for those objects. A critical feature ofthe presently claimed invention is that the prioritization ofinformation requested over the network allows for tremendous increasesin data transmission and processing efficiency. The end result is a morefluid, interactive experience for the user.

At some point after a request is made, information about the requestedperceivable objects arrives at the user's computer from the network. Atthis point, the perceivable object that requested this information hasits manifest flag cleared, and the new perceivable object's manifestflags are set. These new perceivable objects represent a more detaileddescription of the requesting perceivable object. At this point, themain perceivable object is presented to the user again, to reflect theadditional detail that has been received.

Perceivability

The above operations are applied to perceivable objects in a virtualreality world to allow users to move around in a boundless world havingan unlimited amount of detail. The world contains objects that arevisible, audible, tactile, or perceivable in any way.

Each perceivable object's sub-objects must represent a more detaileddescription of that perceivable object. For a visual perceivable objectthat is represented by a simple polyhedron, the sub-objects are eachsmaller polyhedra that, when combined, form a more detailedrepresentation of the original perceivable object. For an auditoryperceivable object that is represented as digital sound sample, thesub-objects are higher-quality representations of the sound. For atactile perceivable object that is represented as a two-dimensionalheight map, the sub-objects are smaller, more detailed height maps that,when put together, comprise a more detailed version of the originaltactile map.

The perceivability of a three-dimensional visually perceivable object isits degree of visibility. A useful way to measure a visually perceivableobject's degree of visibility is by calculating the projected area thatthe object occupies in the user's field of view. This measurement isimproved by setting the visually perceivable object's perceivabilityvalue to zero if the visually perceivable object is either completelyhidden by other visually perceivable objects or is completely outside ofthe user's visual field. The perceivability measurement is refined byconsidering a visually perceivable object's distance from the center ofthe user's visual field, the perceivable object's distance from theuser, its brightness, and its relative velocity, as objects can blurwhen in motion. Advanced implementations of the visibility functionconsider not only the current visibility of a perceivable object, butalso take into account predictions of future visibility.

The perceivability measurement of an audible perceivable object is itsaudibility. This audibility is determined by the user's proximity to thesource of the sound. This measurement is refined by considering thesound wave deflection effects of visually perceivable objects which mayblock the sound, the masking effects of other, louder sounds that theuser is currently hearing, and the relative velocity and direction ofmovement of the audible perceivable object (i.e., the source of thesound) to account for the Doppler effect. As with visually perceivableobjects, advanced implementations consider predictions of futureaudibility.

The perceivability measurement of a tactile perceivable object isdetermined by whether the user is currently or is soon likely to betouching the tactile perceivable object.

All of these perceivability measurements must be weighted appropriatelyso that they can be compared against each other. The weight thatdifferent types of perceivability are given depends on the user'spreferences, both predefined and queried, as well as on the ability ofthe user's computer to present the different types of information.

Detail Threshold

The detail threshold value controls how much information will berequested from the network. If this threshold value is set too high,then not enough detail will be requested and the user will not be takingadvantage of the network's full capacity. If the threshold value is settoo low, a great deal of information will be requested from the network,but the user's computer may not be able to present it fast enough.Therefore, the threshold value is dynamically set to the point where thecomputer is just able to process and present all of the data that itreceives.

FIG. 2 is an example of how visible objects are presented to the user,in this case, what happens as a user approaches a house in the distance.As shown in the figure, the level of detail about a given perceivableobject that is conveyed to the user varies depending upon the object'sperceivability, in this case the visual field area that the perceivableobject occupies. This example shows a user's changing view as the userapproaches one or two houses. The box around each frame represents theuser's field of view in each frame. Both houses are stored on a remotecomputer on the computer network and the user has never before viewedthese houses in detail. Note that as the user approaches the foremosthouse in the scene, the number of polygons in the scene remains constantat about six or seven.

Frame A: Initially, both houses are in the distance and are notdetailed. There are six polygons (perceivable objects) visible in thescene.

Frame B: As the user approaches the closer of the two houses, theperceivability values of both the base of that house and the roof ofthat house exceed the threshold value for those objects and both objectssend messages across the network requesting more detail aboutthemselves. In response to these requests, data coding for the door andwindow are transmitted from the computer where the house is stored andare received by the computer running the application (this may be thesame computer) and are displayed in the scene. At the same time, whenthe user moved closer to the first house, the more distant housedisappeared from the user's field of vision. This more distant house wastherefore removed from the display tree and is no longer represented inthe scene. Thus, the number of perceivable objects in the scene remainsconstant at six.

Frame C: As the user gets even closer to the first house, the doorbecomes large enough to surpass the detail threshold value for thatobject and it requests additional detail about itself. However, thedetail on the door is a window, and that window is the same as thewindow on the second floor. Therefore, a copy of that second floorwindow is created and displayed on the door. No network activity isnecessary. The addition of the window to the door increases the numberof visible perceivable objects in the scene to seven.

Frame D: As the user approaches the door, the window in the doorsurpasses the detail threshold value for that object and it requestsmore detail about itself from the network. Data coding for the glasspanes in the window are transmitted from the computer that owns thewindow to the computer running the application (again, this could be thesame computer) and are presented to the user. The rest of the door alsosurpasses the detail threshold value for that object and it requests,then receives, additional data about itself, in this case, the data thatcodes for the doorknob. The roof and one of the walls of the house arenow outside the user's field of vision and those objects are deletedfrom the display tree. For this reason, the window on the second floordoes not request more detail about itself. Because those additionalperceivable objects were deleted from the display tree, there are stillonly seven perceivable objects visible in the scene.

Propagation of Data Objects

As discussed above, a the data storage and retrieval system of thepresent invention propagates perceivable objects around the network inorder to solve the problem of server overload. There are no servers orclients in the distributed system; each computer in the system runs aprogram that acts as a node. FIG. 3 shows perceivable object propagationamong three computers.

In FIG. 3, Computer 1 has requested a perceivable object from Computer3. Computer 2 requests that same perceivable object from Computer 1instead of Computer 3, because its network connection to Computer 1 isfaster. An example of when this situation would occur is if Computers 1and 2 are in the same local area network, and Computer 3 is in adifferent country.

Application to Three-dimensional Worlds

These concepts of perceivable objects and messages, including selectivedata retrieval (based on the perceivability of perceivable objects andtheir attendant detail threshold values) and propagation of perceivableobjects across the network, may be used in a data storage and retrievalsystem of the present invention to create a large and detailedthree-dimensional world from data distributed across a computer network.Users would access this world by means of an executable computer programresident on their hard disk. When a user runs the application program,the world would appear on the user's computer screen.

The user would see buildings and trees and other real-world perceivableobjects in the three-dimensional environment. The user would also seethree-dimensional graphical representations of other users currently inthe world and would be able to talk and interact with those other users.Clubs could exist in the world for users to meet other people withsimilar interests. Users could be able to play games, such as chess orsimulated combat with each other. Companies could open “stores” in theworld where they could sell products to interested users, and spacecould be allocated in which companies could advertise products andservices to the users.

The contemplated world could contain graphical representations ofinformation currently available on the World Wide Web, such as librariesand painting galleries. In addition, new information that currentlycannot be represented well in the two-dimensional environment of theWorld Wide Web, such as galleries of sculptures and full scale models ofancient and modern cities, can be presented to the user in aninteresting and enjoyable manner.

The data storage and retrieval system of the present invention can beused to immerse a user in a large and detailed virtual reality worldwithout sending the entire world over the network, so that the user caninteract smoothly with perceivable objects in the virtual reality worldand with other users. In order to accomplish this objective, the methoddistributes the world about the network and sends only the relevantportions of the world to the user.

Any given user may interact with other users and with perceivableobjects in the virtual reality world. Whenever a user interacts withanother user or manipulates a perceivable object, e.g., by selecting theobject with an aural command, touchscreen command, cursor selection,mouse selection, trackball selection, joystick selection, keystrokeselection, or stylus selection in order to request additional detailabout the object, the priority of the other user or the objectincreases. In addition, a user that owns certain objects presented inthe world may grant various levels of access privileges to one or moreother users. A messaging system handles the interaction between the userand perceivable objects stored on remote computers.

The user interface enables the user to interact with objects and withother users of the virtual world. The specific ways that the user mayinteract with the virtual world are determined by the user interface.Examples of such interactions are textual, aural, and/or visualcommunication between one or more users in the world, moving and copyingperceivable objects, purchasing actual products represented byperceivable objects, selecting perceivable objects—including digitalrepresentations of other users—with a mouse or cursor command, andmanipulating perceivable objects, such as opening and closing doors. Theuser interface allows users to employ these methods of interaction andto effectively interact with and move about the world.

The user interface enables users to create new perceivable objects whilethey are in the world instead of limiting users to importing perceivableobjects into the world. Users and designers will then be able to createnew perceivable objects by graphically moving, copying, and modifyingother perceivable objects in the world. All of this creation will becarried out while in the three-dimensional virtual environment.

Animation may be provided so that perceivable objects can move on theirown. The digital representations of people in the world will then beable to walk, doors will open and close smoothly, and the world will bemore realistic.

Creating a Three-dimensional World

The foundations for a large and detailed world will then be createdusing the application program. The perceivable objects in the world willconsist of realistic perceivable objects such as trees, mountains,houses, stores, and lakes, as well as abstract perceivable objects suchas World Wide Web documents. Objects can be represented realistically orwith a surreal scheme. Both representations will be tested for visualappeal and ease of creation.

Physical Objects

Named perceivable objects may be created and both their geometricdescriptions and their sub-objects may be specified. Users with theappropriate authorization can create new perceivable objects and canmanipulate existing perceivable objects. In this way, users can use theapplication program itself to build new perceivable objects in theworld.

When designing complex perceivable objects, users may want to use adedicated three-dimensional design program, such as AutoCAD. Tofacilitate this process, as well as to allow the use of existingthree-dimensional designs, a method of converting files from theseprograms into the application program's native markup language isprovided. This conversion is difficult because the perceivable objectsin the foreign file must be organized into a hierarchical structure.Still, there are known methods to create this hierarchical structure andthese methods can be implemented with sufficient efficiency. The methodthat will most probably be used was developed by J. Goldsmith and J.Salmon and is described on page 707 of “Computer Graphics, Principlesand Practice” by James Foley.

To aid in world design, a library of commonly used geometric structuresmay be developed. These structures may be stored locally on each user'scomputer. The perceivable objects in this library can be put togetherinto more complex perceivable objects. This library greatly simplifiesthe creation of new virtual reality worlds. The markup language caneasily interface with these perceivable objects so an author can work ata high level of abstraction when designing. Also, because theseperceivable objects are stored locally, they lessen network traffic.

Some common structures cannot be represented well by static geometricperceivable objects. For example, stairs are quite common and are quitetedious to construct geometrically. Procedure based perceivable objectscan help solve this problem. For example, a relatively simple operationmay be performed which, when given certain values like steepness andnumber of stairs, creates a geometric description of a staircase.Procedural perceivable objects contain operations such as the staircaseoperation and make construction of a world much simpler. If theseprocedures are stored locally on each user's computer, the values forthe staircase can be sent across the network instead of its geometricdescription. The parametric description is much more compact, so lessnetwork bandwidth is used.

Abstract Objects

In order to present the user with information that is not inherentlyphysical, abstract perceivable objects can be created. Each abstractperceivable object takes in data of a specific type and creates athree-dimensional representation of that data. The data can be text,images, or any other data for which a physical representation can beconceived. The processing necessary to convert data into athree-dimensional form can be carried out on a remote computer acting asa gateway. For commonly used types of abstract data whose processing isnot excessively expensive, the processing can be carried out by thelocal computers.

A great amount of information on the Internet is in the World Wide Web'sHTML format, so a great deal of content can be added to the virtualreality world by interfacing to this format. HTML does not inherentlytranslate to a three-dimensional representation because the web of linksemanating from a document have no physical counterpart. Still, sometypes of spatial representations of the world wide web are conceivableand can be implemented with the application program. Theserepresentations can ease the difficulty of locating information on theWorld Wide Web.

Most of the information in the physical world is stored as textdocuments. These documents themselves have no physical representationother than the characters that compose them, so the application programcannot greatly improve the presentation of the information itself. Theapplication program can, however, make it easier for users to find theinformation that they want by presenting users with an organizationalscheme with which they are already comfortable and experienced: alibrary of books. Using the application program, users can walk througha virtual library that is organized in the same way as a conventionallibrary.

User Interaction

An interactive world is made up of many perceivable objects which can beacted on in useful ways. The application program must therefore providea user interface with which the user can perform these actions. Becausethe program seeks to immerse the user in the virtual reality world, theuser interface must be easy to use. It must provide the user withcontrols to navigate through the virtual reality world and interact withthe perceivable objects in this world.

Often, specialized hardware is used to allow interaction with a virtualreality world. Because the application program maintains athree-dimensional model of the world that can be displayed from anyviewpoint, it is easily interfaced with whatever virtual realityhardware the user may have.

The most basic feature that the user interface provides is to allowusers to move around in the virtual reality world. Input devices on adesktop computer can be used in various ways to move a virtual bodythrough a three-dimensional world. Current virtual reality computergames already incorporate many of these interface mechanisms. Theapplication program will allow the user to select a comfortablenavigational interface from a list of several of these well establishedmechanisms.

Although walking may seem the most natural for users in the beginning,the convenience of flight makes it the preferred conveyance in theapplication program. For some purposes, the user may wish to travel in avirtual vehicle such as a spaceship, a tank, or even a submarine.Because of the true three-dimensional representation that theapplication program maintains, such customized interfaces and movementconstraints are quite easily implemented.

In order to interact with a perceivable object in the virtual realityworld, the user must first specify the perceivable object on which he orshe wants to act. The easiest way to select a perceivable object is topoint at and click the perceivable object using the mouse. However, theexact perceivable object the user is specifying can be ambiguous. Forexample, if the user clicks the door of a house, it is unclear whetherhe intends to indicate the door or the house. In some cases, theambiguity can be resolved by considering what the user wants to do tothe perceivable object. If the user is trying to open the perceivableobject, then it is clear that the user meant to select the door, becausethe house cannot be opened. If, however, the user attempts to paint theperceivable object, the application program must explicitly ask the userwhich perceivable object is intended.

The user must also have a mechanism of specifying what action is to beperformed on the selected perceivable object. The user will make thischoice from a menu of actions that are allowed on the selectedperceivable object. For convenience, perceivable objects will havedefault methods, so that the user can open a door or read a book simplyby double-clicking on them. These shortcuts will make the most commonactions readily accessible to the user.

Once the user has selected a perceivable object and the action to beperformed on that perceivable object, the application program promptsthe user for any additional values that the action supports. A messageis then assembled with the desired action and values. This message isthen sent to the local copy of the selected perceivable object. If thelocal computer does not own the selected perceivable object and cannothandle the message, the message is sent across the network to thecomputer that owns the perceivable object. E.g., a message to open adoor must be sent to the computer that owns the door, since a new stateof being must be created for the door, while a message to examine theclosed door in greater detail can be handled by the local computer,since the local computer already has all of the data relative to thedoor in its closed state.

It will now be apparent to those skilled in the art that otherembodiments, improvements, details and uses can be made consistent withthe letter and spirit of the foregoing disclosure and within the scopeof this patent, which is limited only by the following claims, construedin accordance with the patent law, including the doctrine ofequivalents.

What is claimed is:
 1. A computer network data distribution andretrieval system comprising: A) at least two microprocessors connectedsuch that the microprocessors are in communication, forming a network;B) at least one data source having data representing at least one objectdistributed between said at least two microprocessors in said networksuch that i) said data is adapted for categorization into hierarchicaldetail levels; and ii) there exists a method of measuring aperceivability value for said at least one object; C) a firstapplication program operating on at least one of said microprocessorsfunctioning such that i) said at least one microprocessor acts as anetwork node, ii) said application program controls data retrieval fromsaid at least one distributed data source by a) measuring aperceivability value for said at least one object b) assigning apriority status to said at least one object based on said measuredperceivability value, c) comparing said priority status for said atleast one object with a predetermined or dynamically varied thresholdvalue, d) instructing said at least one object to request additionaldetail about said at least one object when said priority status equalsor exceeds said predetermined detail threshold value, e) ordering saidrequests for additional detail based on the priority of said at leastone object relative to the priority of any other object requests, and f)sending the first-ordered request across the network; and D) a useroperating at least one of said microprocessors.
 2. The computer networkdata distribution and retrieval system, as claimed in claim 1, whereinthe perceivability value measured relates to the visibility of said atleast one object.
 3. The computer network data distribution andretrieval system, as claimed in claim 2, wherein the visibility of saidat least one object is measured by calculating the visual field areaoccupied by said at least one object.
 4. The computer network datadistribution and retrieval system, as claimed in claim 2, wherein theperceivability value of said at least one object is set to zero if thevisually perceivable object is either completely hidden by othervisually perceivable objects or is completely outside of the user'svisual field.
 5. The computer network data distribution and retrievalsystem, as claimed in claim 2, wherein the perceivability value of saidat least one object is scaled by considering the distance of said atleast one object from the center of said user's visual field, thedistance of said at least one object from said user, the brightness ofsaid at least one object, and the velocity of said at least one objectrelative to other objects.
 6. The computer network data distribution andretrieval system, as claimed in claim 2, wherein the perceivabilityvalue of said at least one object is determined by not only the currentvisibility of said at least one object, but also by predictions of thefuture visibility of said at least one object.
 7. The computer networkdata distribution and retrieval system, as claimed in claim 1, whereinthe perceivability value measured relates to the audibility of said atleast one object.
 8. The computer network data distribution andretrieval system, as claimed in claim 7, wherein the audibility of saidat least one object is measured by calculating the user's proximity tothe source of the sound.
 9. The computer network data distribution andretrieval system, as claimed in claim 7, wherein the perceivabilityvalue of said at least one object is set to zero if the audiblyperceivable object is either completely hidden by other perceivableobjects or is completely outside of the user's audible field.
 10. Thecomputer network data distribution and retrieval system, as claimed inclaim 7, wherein the perceivability value of said at least one object isscaled by considering the distance of said at least one object from thecenter of the user's audible field, the distance of said at least oneobject from the user, the loudness of said at least one object, and thevelocity of said at least one object relative to other objects.
 11. Thecomputer network data distribution and retrieval system, as claimed inclaim 7, wherein the perceivability value of said at least one object isdetermined by not only the current audibility of said at least oneobject, but also by predictions of the future audibility of said atleast one object.
 12. The computer network data distribution andretrieval system, as claimed in claim 1, wherein the perceivabilityvalue measured relates to the tactility of said at least one object. 13.The computer network data distribution and retrieval system, as claimedin claim 12, wherein the tactility of said at least one object isdetermined by whether said user is currently or is soon likely to betouching the tactually perceivable object.
 14. The computer network datadistribution and retrieval system, as claimed in claim 1, wherein saidat least one object has more than one measurable perceivability value.15. The computer network data distribution and retrieval system, asclaimed in claim 14, wherein said more than one measurableperceivability value are weighted so that they can be compared againsteach other.
 16. The computer network data distribution and retrievalsystem, as claimed in claim 15, wherein the weight of each said morethan one measurable perceivability value is determined by said user'spreferences.
 17. The computer network data distribution and retrievalsystem, as claimed in claim 16, wherein said user's preferences arepredefined.
 18. The computer network data distribution and retrievalsystem, as claimed in claim 16, wherein said user's preferences aredefined by said user in response to a query from said applicationprogram.
 19. The computer network data distribution and retrievalsystem, as claimed in claim 16, wherein said user's preferences are alsodetermined by the ability of said user's computer to present thedifferent types of information.
 20. The computer network datadistribution and retrieval system, as claimed in claim 1, wherein thepriority status of said at least one object is determined by userinterest.
 21. The computer network data distribution and retrievalsystem, as claimed in claim 20, wherein user interest is determined by amethod selected from the group consisting of aural command, touchscreencommand, cursor selection, mouse selection, trackball selection,joystick selection, keystroke selection, and stylus selection.
 22. Thecomputer network data distribution and retrieval system, as claimed inclaim 1, wherein said application program controls data retrieval fromsaid distributed data sources by instructing said at least one object todiscard said additional detail about said at least one object when saidpriority status becomes less than said predetermined detail thresholdvalue.
 23. The computer network data distribution and retrieval system,as claimed in claim 1, wherein the system further comprises a secondapplication program operating on at least one other said microprocessorsuch that said second application program responds to requests foradditional detail received from said first application program.
 24. Thecomputer network data distribution and retrieval system, as claimed inclaim 23, wherein said requests for said additional detail are orderedbased on the priority of said at least one object relative to thepriority of other objects that have sent requests.
 25. The computernetwork data distribution and retrieval system, as claimed in claim 24,wherein the responses are ordered and sent across the network accordingto the priority of said request.
 26. The computer network datadistribution and retrieval system, as claimed in claim 24, wherein theresponse is sent across the network regardless of the priority of saidrequest.
 27. The computer network data distribution and retrievalsystem, as claimed in claim 1, wherein said priority by which data issent is determined by both said perceivability value and by a user'sinteraction preferences.
 28. The computer network data distribution andretrieval system, as claimed in claim 27, wherein said preferences canbe defined by a user.
 29. The computer network data distribution andretrieval system, as claimed in claim 1, wherein said priority by whichdata is sent is determined by said perceivability value and the abilityof said at least one microprocessor to present the different types ofinformation.
 30. The computer network data distribution and retrievalsystem, as claimed in claim 29, wherein said ability of said at leastone microprocessor to present the different types of information can bedefined by a user.
 31. The computer network data distribution andretrieval system, as claimed in claim 1, wherein for prioritycalculation, said perceivability values are scaled appropriately so thatsaid priorities can be compared.
 32. The computer network datadistribution and retrieval system, as claimed in claim 1, wherein thethreshold value is dynamically set to the point where said at least onemicroprocessor is just able to process and present all of the data thatit receives.
 33. The computer network data distribution and retrievalsystem, as claimed in claim 1, wherein at least one other object may becreated by a user.
 34. The computer network data distribution andretrieval system, as claimed in claim 33, wherein said user may grant toone or more other users various levels of access privileges to said atleast one other object.
 35. The computer network data distribution andretrieval system, as claimed in claim 1, wherein at least one object maybe manipulated by a user.
 36. The computer network data distribution andretrieval system, as claimed in claim 35, wherein manipulation of saidat least one object increases the priority of that object.
 37. Thecomputer network data distribution and retrieval system, as claimed inclaim 35, wherein manipulation of said at least one object is achievedby a method selected from the group consisting of aural command,touchscreen command, cursor selection, mouse selection, trackballselection, joystick selection, keystroke selection, and stylusselection.
 38. The computer network data distribution and retrievalsystem, as claimed in claim 1, wherein at least one of saidmicroprocessors is provided with means for presenting said at least oneobject.
 39. The computer network data distribution and retrieval system,as claimed in claim 38, wherein the perceivability of said at least oneobject is measured by the difference between the perception of theobject when it is presented and the perception without the object beingpresented.
 40. The computer network data distribution and retrievalsystem, as claimed in claim 1, wherein said object is presentedvisually.
 41. The computer network data distribution and retrievalsystem, as claimed in claim 2, wherein the visibility of said at leastone object is measured by the visual difference between the scene withthe object presented and without the object presented.
 42. The computernetwork data distribution and retrieval system, as claimed in claim 40,wherein said visual difference is determined with reference to definedpreferences.
 43. The computer network data distribution and retrievalsystem, as claimed in claim 2, wherein the visibility of said at leastone object is measured by the decrease in deviation of occupied screencoordinates between said at least one object, together with any siblingsof said at least one object, and the parent object.
 44. The computernetwork distribution and retrieval system, as claimed in claim 3,wherein the visual field area occupied by said at least one object iscalculated by means of determining the display screen coordinatesoccupied by said at least one object.
 45. The computer network datadistribution and retrieval system, as claimed in claim 1, wherein saidobject is presented to said at least one user aurally.
 46. The computernetwork data distribution and retrieval system, as claimed in claim 7,wherein the audibility of said at least one object is measured by theaudible difference between the audible field with the object presentedand without the object presented.
 47. The computer network datadistribution and retrieval system, as claimed in claim 1, wherein saidobject is presented to said at least one user tactilely.
 48. Thecomputer network data distribution and retrieval system, as claimed inclaim 12, wherein the tactility of said at least one object is measuredby the tactile difference between the object being presented and theobject not being presented.
 49. The computer network data distributionand retrieval system, as claimed in claim 1, wherein said detail of saidat least one object is represented by at least one separate object. 50.The computer network data distribution and retrieval system, as claimedin claim 49, wherein said at least one separate object is presented inplace of said at least one object.
 51. The computer network datadistribution and retrieval system, as claimed in claim 49, wherein saidat least one separate object is added to said at least one object. 52.The computer network data distribution and retrieval system, as claimedin claim 1, wherein said detail of said at least one object isrepresented by additional data that does not represent at least oneseparate object.
 53. The computer network data distribution andretrieval system, as claimed in claim 52, wherein said additional datarefines said at least one object without replacing said at least oneobject.
 54. The computer network data distribution and retrieval system,as claimed in claim 23, wherein said requests for additional objectdetail are queued in the order in which they were received and responsesare generated likewise.
 55. The computer network data distribution andretrieval system, as claimed in claim 24, wherein a response isgenerated for each request and sent to said first application programaccording to the priority status of said at least one object relative tothe priority of other objects that have already sent requests foradditional object detail.
 56. The computer network data distribution andretrieval system, as claimed in claim 24, wherein said response is sentto said first application program on receipt by said second applicationwithout reference to the priority status of said at least one objectrelative to the priority of other objects that have already sentrequests for additional object detail.
 57. The computer network datadistribution and retrieval system, as claimed in claim 27, wherein saidinteraction preferences are defined in response to a query from saidapplication program.
 58. The computer network data distribution andretrieval system, as claimed in claim 27, wherein said interactionpreferences are limited by the ability of said at least one of saidmicroprocessors operating said first application program to present saidobject.
 59. The computer network data distribution and retrieval system,as claimed in claim 1, wherein said priority by which data is sent isdetermined by said perceivability value and the ability of said at leastone of said microprocessors operating said first application program topresent said object.
 60. The computer network data distribution andretrieval system, as claimed in claim 1, wherein said ability of said atleast one of said microprocessors operating said first applicationprogram to present said object can be defined.